Augmented reality (AR) relates to providing an augmented real-world environment where the perception of a real-world environment (or data representing a real-world environment) is augmented or modified with computer-generated virtual data. For example, data representing a real-world environment may be captured in real-time using sensory input devices such as a camera or microphone and augmented with computer-generated virtual data including virtual images and virtual sounds. The virtual data may also include information related to the real-world environment such as a text description associated with a real-world object in the real-world environment. The objects within an AR environment may include real objects (i.e., objects that exist within a particular real-world environment) and virtual objects (i.e., objects that do not exist within the particular real-world environment).
In order to realistically integrate virtual objects into an AR environment, an AR system typically performs several tasks including mapping and localization. Mapping relates to the process of generating a map of a real-world environment. Localization relates to the process of locating a particular point of view or pose relative to the map of the real-world environment. In some cases, an AR system may localize the pose of a mobile device moving within a real-world environment in real-time in order to determine the particular pose associated with the mobile device that needs to be augmented as the mobile device moves within the real-world environment.
An AR environment may be provided to an end user (also referred to more generally as a user) of a mobile device using an electronic display (e.g., an LED display integrated with a head-mounted display device). The electronic display may display images of virtual objects to the end user by modulating light provided to the electronic display (e.g., a liquid crystal on silicon display) or by generating light within the electronic display (e.g., an OLED display).
In order to generate a realistic AR environment it is important to achieve low latency, which in part can be achieved by increasing a frame rate. For example, for an end user wearing a head mounted display (HMD) device, if too much time lapses between the time the end user's head turns away from a particular pose and the time an image of a virtual object is displayed based on the particular pose, then the virtual object will appear to drift away from its intended location within the AR environment. For example, the image may not appear to be aligned with an intended real-world location or object, which is undesirable.